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ABSTRACT 

A study was conducted to review and organize existing 
literature on the evaluation of communication skills training 
programs. When setting training objectives, four levels of evaluation 
should be considered: (1) reaction (how did participants feel about 
the training?); (2) learning (what did the participants learn from 
the training?); (3) behavior (how has the learning been applied at 
work?); and (4) results (how has the training affected the 
organization?) • The paper provides an overview of 19 published 
evaluations (discussed in alphabetical order by name of author) which 
were studied in hopes of encouraging and directing future evaluation 
efforts. Results of the study provided a basis for soveral 
recommendations to guide future evaluation efforts. First, 
evaluations should be multi- level. Second, trainers and researchers 
should continue using a variety of evaluation methods to strengthen 
research findings. Third, techniques for accurately measuring 
behavior changes need to be developed. Fourth, more studies to 
determine the training's ultimate impact on the organization ought to 
be developed. Fifth, the long-term effects of training must be 
examined. Finally, evaluations should be published and serve as 
models for future efforts. (One table is included, and 39 references 
are actachecl.) (MG) 
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INTRODUCTION 



Amidst the transfomiation into a high-tech, service- oriented 
society, businesses are coming to new realizations. No longer can 
a conpany expect to b^aat the competition without state-of-the-art 
technology. And row, no longer can a company expect to beat the 
ccmpetition without state-of-the-art people. As businesses 
increasingly recognize the need for competent employees to provide 
the quality of service society demands, the response to this need 
seems to be ever more frequently "human resource training." 

Human resource training is seen as a viable way of iinproving 
organizational effectiveness by improving the ability of 
individual employees to perform their jobs. In recognition of tlL'.' 
benefits of human resource training, organizations are spending 
increasing amounts of money on it. In 1985 the United States 
corporatic^s and governments spent an estimated $215 billion to 
train eniployees. This is nearly as much as the total aniount spent 
on primary, secondary, and higher education for the same tit-e 
period (Carnevale, 19&6} . 

Importance of Caimunication Training 

Among the most prevalent content areas for hutiian resource 
training is ccnmunication (Johnson, 1986). In 1986, an estimated 
66.2% of organizations with 50 or more employees provided seme 
fonti of conmunication skills training (Norback, 1987) . The 
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importance of contiiunication in business and industrial settings 
has long been recognized, so it should not be surprising that 
organizations are willing to invest in cGnmunication skills 
training." Simply put, organizations believe that by training 
individuals to ccn«iunicate better, they are improving the 
efficiency and effectiveness of the entire organization. 

An investment in a training program may be viewed as an 
investment in a set of procedures designed to bring about a 
; relatively permanent change in a person's behavior as a result of 

practice (Wexley & Latham, 1981) . Canmunication skills training 
is often aimed at improving interpersonal skills, group 
interaction:., supervisory caritiunication, listening skills, and 
other behaviors. "[Conmunication training] should not aini to make 
the participants aware , or sensitive , or even knowledgeable , 
rather it should leave them with the ability to car«iunicate 
better" (Anastasi, 1987, p. 741). When organizations invest in 
cowtiunication skills training, they expect results. They expect 
positive, visible changes in behavior and they ultinjately expect 
an ifftpact on organizational effectiveness. 

Need for Training Evaluations 

With ths growing emphasis — and dollars — placed on training in 
organizations, it would seem logical, if not imperative, that 
training professionals conduct formal training evaluations to 
ensure that expectations are being met. However, relatively few 
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evaluations are actually conducted (Bunker & Cohen, 1978; 
aieltzer, 1979) . One survey of trainers indicated training 
evaluation is considered to be the most critical issue in their 
field, yet 90% of the respondents had no definite method for 
evaluating training (Olivas & Inman, 1983} . This is an 
unfortunate fact because when money gets tight, programs which 
cannot be accounted for will invariably be cut, and training 
departments which do not substantiate their contribution to. their 
companies are usually among the first to experience cutbacks 
(Zenger & Hargis, 1982). 

If training professionals recognize the importance of 
evaluation, why are evaluations so rare? One of the main reasons 
why evaluations are not conducted is because they are not required 
by managanent. Seme trainers have reported that their managers 
actually discourage evaluation with, "We wouldn' t have hired you 
if we didn't think you were effective." Several studies have 
indicated trainers' reluctance to "waste tinie" testing something 
managers have convinced themselves is good (Carlisle, 1984) . 
However, in lean economic times, these same training progranis may 
be viewed as superfluous (Bell & Kerr, 1987) . As long as the task 
of evaluating the cost-effectiveness of training is neglected, 
training may not be seen as a budget priority. 
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Fears Surrounding Training Evaluations 

There are several fears which may surround coifTiunication 
training evaluation. One fear is that ccnstiunication training 
results may be too difficult to treasure. A training professional 
may wonder, "What if I could see the effects of my program, but my 
efforts to measure those effects failed? How am I supposed to 
prove the program really was worthwhile?" This trainer probably 
does not know that different evaluation techniques have been 
developed to detect effects at a variety of levels. 

Another fear may evolve around limited resources, A trainer 
who has read about evaluations which involved simulating realistic 
settings to test learning (e.g., Bennett, 1981; Bowers, Gilchrist, 
& Browning, 1980; Clark, et al., 1985; Moses & Ritchie, 1976), or 
evaluations which entailed conducting series of follow-up 
interviews (e.g., Byham, et al., 1976; Hand & Slocum, 1970) may be 
fearful that the require^ients in time, energy and money are too 
steep. This trainer would need to be exposed to the various 
alternative evaluation approaches which are less expensive and 
less time-consuming. 

Finally, people may fear that they are incapable of 
conducting the evaluations, especially if they have limited 
research backgrounds. Rather than avoiding evaluation altogether, 
these people should seek advice front consultants in organ3.zational 
caiiriunication or industrial/organizational psychology to handle 
the most coniplicated procedures. Plus, there are seme simple 
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evaluation methods which they could learn on their ovjn. 

Most trainers recognize the importance of training 
evaluations, but some possess very real fears about measurotient 
methods. It is time to put those fears aside. As long as 
coitmuni cation training programs are not evaluated, there is no way 
of knowing whether they are meeting the organizations' needs and, 
ultimately, whether they are worth investing in. 



Purpose of this Study 

The purpose of this study is to review and organize existing 
literature on the evaluation of coritiunication skills training 
programs. This study will examine a sample of previous efforts in 
hopes of encouraging and directing future evaluation efforts. 

The remainder of this report will be organized as follows: 
First, the four levels of evaluation will be discussed as they 
pertain to communication skills training. Next, previous 
evaluations of ccromunication training prograiTis will be reviewed. 
Finally., implications of this study for future research in human 
resource development will be considered. 
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LEVELS OP EVALUATION 

Evaluations are conducted for one obvious reason: to 
deterniine the effectiveness of training progranis. What 
constitutes "effectiveness" in ccmitunication training, however, is 
not so obvious. Usually in organizations a project is considered 
effective if it results in sufficient monetary gains. 
Unfortunately, it is n^t always easy to assign a dollar value to 
training results, especially for some of the intangible benefits 
of ccxTfTiunication training. The preferred criteria for measurement 
of coKuunication training effectiveness should be specific 
training objectives. However, most training programs are not 
evaluated on whether training objectives are achieved. As a 
result, organizations do not know whether they should continue, 
modify, or abandon training efforts (Latham, 1982j . 

There are four levels of evaluation to consider when setting 
training dsjectives. Each of these levels can be defined as 
follows: 

1) Reaction How did participants feel about the training? 

2} Learning Mhat did participants learn from the training? 

3) Behavior How has the learning been applied at work? 

4} Results How has the training affected the organization? 
In this paper, the pertinent issues surrounding coHiiunication 
training evaluation at all of these levels will be explored. Each 
level will be briefly described, then variables, methods and 
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special considerations v;ill be discussed. 

r; Reaction 

In most organizations training prograiTis are only evaluated at 
the reaction level. The usual measurement tool is an end-of- 
course questionnaire or "happiness index" designed to find out how 
much participants liked the workshop (Del Gaizo, 1984) . The 
questionnaire is scmetimes jokingly referred to as the "smile 
sheet" under the notion that scores are related to how often the 
instructor smiles (Fitz-enz, 1984] . In a way, this evaluation can 
be looked at as a "customer satisfaction" measure (Kirkpatrick, 
1987). The goal should be to find out whether or not participants 
were satisfied with the program. 

SoTie people dismiss evaluations on this level as meaningless 
since there is no assurance that learning has taken place or that 
behaviors will change. However, participants' reactions should be 
taken seriously because they can provide useful information for 
iittproving the training experience. In addition, it is not 
uncaimon for management to make decisions about a training 
prograiii's future based on the ccnments . f one or two participants. 

Variables. Scxne variables which are frequently considered in 
reaction evaluations are course content, instructional approach, 
instructor skills, training cliniate, and overall impressions. 
Under course content, we could examine feelings about the course 
objectives, materials, and information covered. The instructional 
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approach (i.e., behavioral modeling, lecture, discussion) could be 
evaluated in tenrts of perceived effectiveness for delivering the 
course material, h few of the instructor skills which can be 
included in the evaluation are presentation, clarity, and 
responsiveness to participants. One other very important, yet 
often neglected, instructor consideration is how wBll the 
instructor serves as a model of the ccxicnunication skills being 
taught. As training consultant Arty Trost has ronarked, "It's 
amazing how often conmunication trainers don't practice good 
ccniriunication skills" (1985) . 

The training cliniate can be evaluated in terms of the 
following necessary conditions: a climate of mutual respect, a 
cluiiate of collaborativeness rather than competitiveness, a 
climate of support iveness rather than judgmentalness, a climate of 
mutual trust, a cliiriate of fun, and a hunian climate. These 
conditions are recognized by adult education expert Malcolm 
Knowles as most conducive to learning (1987) . 

Finally, the overall impressions of the course should be 
considered. Some relevant topics include the perceived value or 
importance of the course and whether a participant feels the 
learning can be applied in the workplace. 

Methods. The most popular method for measuring reaction is 
the end>of -course questionnaire. I': is generally favored because 
it is short and simple and the results can be easily quantified 
for statistical analyses. Examples of published comrtunication 
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training studies which employed this method are Bell & Kerr 
(1987) , BTiener & Rye (1975) , and McNamara and associates (1982) . 

There are alternative, qualitative methocls for measuring 
reactions, Elliott (1978) used a subjective, written reaction. 
Hand & Slocum (1970) obtained reactions through exit interviews 
conducted with participants after the training. Final 3.y, Cof&ian 
(1979) conducted meetings with "key clients," those people 
responsible for trainee productivity on the job, to obtain delayed 
reactions from observers. 

Special Considerations, There are several, considerations to 
make when developing reaction evaluation instrutiients. One is the 
honesty of participants. If participants are allowed to remain 
anonymous, they are more likely to respond candidly. 

Another consideration is the use of a standard or nonstandard 
instrument. A standardized instrurtent vrould have the advantages 
of demonstrated reliability and validity and would allow for 
ccmparisons between different training groups, while a nonstandard 
format would provide more flexibility to measure course 
effectiveness in its own unique context. 

Finally, the timing of the evaluation should be considered. 
Fomiative evaluations are used during training programs (i.e., 
after each module) to provide ongoing feedback to the instructor, 
Surrniative evaluations are usually conducted iintiediately after the 
course has concluded, but at this time participants may be at an 
unusual high and therefore unable to objectively critique the . 
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training. A "delayea reaction" could be obtained a few weeks 
after the training, once the initial excitement has died down. 

Learning 

The learning evaiaation is conducted to determine whether or 
not the training program has been successful in developing 
knowledge or behaviors. Every program should have scirie learning 
objectives which could be evaluated. Typically in cormunication 
courses, these objectives are specific cotwiunication skills. 

The internal validity of training is based on the learning 
evaluation. If the participants cannot dationstrate that they have 
learned the skills or inforniation taught in the prograni, then the 
program is considered invalid. In addition, we must rariOdiber that 
if no learning has taken place, there can be no resulting 
behavioral changes on the job. • 

Variables. There are two basic variables which can be 
measured tlirough a learning evaluation. The first is knowledge. 
In a coffiiiunication skills training prograrrt, a researcher might 
wish to determine whether participants understand sane basic 
conmunication concepts. For example, in a study by Hand and 
Slocuiri U970) , participants were tested on their knowledge of 
human relations. A researcher might also look for a participant's 
ability to discriniinate between different cormunication behaviors 
in a given context (e.g., Elliott, 1978; Eitiener & Rye, 1975; 
McNamara, et al., 1982). 
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The second variable of learning is behaviors. Because 
canriunication skills training is generally oriented toward 
changing behaviors, participants are frequently asked to 
demonstrate or role-^play what they have learned. The focus of 
these evaluations should be on the participant's ability to 
demonstrate learning of active listening, public speaking, or 
whatever other skills the course focuses on. 

Methods. A variety of methods can be used to measure 
learning. These methods should be chosen according to the 
objectives of the course. To measure knowledge gain, some 
canmunication training studies have used standard tests (e.g., 
Hand & Slocum, 1970; McNamara, et al., 1982). Other times, 
participants have been asked to discrimate between effective and 
ineffective comnunication in given situations (e.g., Elliott, 
1978; Diiener & Rye, 1975). 

To nteasure behavioral learning, a series of exercises may be 
used to determine whether participants can demonstrate each of the 
skills taught. The Bell and Kerr study (1987), for exaniple, 
tested each participant's writing, presentation, and dictation 
skills through exercises. Other studies may involve role-play 
tests designed to measure the ability to apply skills in siinulated 
settings (e.g., Bennett, 1981; Bowers, Gilchrist, and Browning, 
1980; Burnaska, 1976; Clark, et al., 1985; Moses & Ritchie, 1976), 
Finally, behavioral learning may be measured through written tests 
with questions asking participants to respond as they would in 



11 



real-life situations (e.g., Elliott, 1978; Eniener & Rye, 1975; 
Siiith [study B] , 1976) . One limitation of written behavior tests 
is that they discount the need for physically daiionstrating 
skills, including nonverbal ccmnunication skills. 

Special Considerations. Most published cortriunication 
training evaluation efforts have eiriphasized evaluation at the 
learning level, but this level is not as important in 
conririunication skills training as the behavior level or the results 
level. Seme comnunication training studies demonstrate that 
learning gain does not necessarily correspond with behavior 
changes at work (e.g., Burnaska, 1976; Bonener & Rye, 1975). So, 
while much focus is applied to detenriining whether participants 
understand coniriunication or understand corrciunication skills, 
perhaps more attention should be given to ensuring that 
participants are able to apply conmunication skills at work. It 
is arguable that simply viewing participants in classroom 
perforntances is all the evaluation that is needed at the learning 
level because it is enough to ensure that learning is taking 
place. 

Another consideration for evaluating at the learning level is 
resources. Many learning studies deniand liberal investments in 
time and money. For example, simulating realistic environnients, 
acting out role-plays, and coding each individual's behavioral 
responses can be very costly. If resources are liniited, other 
methods may be considered more feasible. 
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One more consideration is whether learning evaluation results 
can be attributed to the training. If learning is measured only 
after training, there is no way to tell whether participants might 
have had the knowledge prior to training. Also, if no control 
group is used, a researcher cannot be assured that the 
participants did not gain the knowledge elsewhere. Finally, it is 
important to consider whether testing which occurs before the 
training could have influenced the results of a test after the 
training, as McNamara and associates have (1982 j . 

Behavior 

Evaluation at the behavior level would measure the 
application of training to the workplace. People are often unable 
to transfer the skills they have learned in training sessions onto 
the job. This problem, which is particularly relevant with 
ccnrounication and other people-oriented skills training, should be 
addressed in training evaluations (Trost, 1985) . When training is 
ineffective in producing the behavioral results intended, the 
result may be costly in terms of lost productivity and lost 
confidence in the training function (Kelley, Orgel, & Baer, 1984). 

Variables. Evaluation on the behavior level must correspond 
directly with the training objectives. Sane variables which can 
be evaluated include speaking, writing, listening, group 
cawriunication, interviewing, interdepartmental ccrrniunication, 
nonverbal corwriunication, dictating, rtieeting facilitation, sales 
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presentations, supervisory coiifliunication, assertive conntiunication, 
and many others, depending on the nature and objectives of the 
:raining. 

Methods. Behavior changes are most conirionly measured via 
questionnaires. Some communication skills training studies have 
asked trainees to rate their ovm behaviors through questionnaires 
(e.g., Bell & Kerr, 1987; Blakeslee, 1982; Einener & Rye, 1975; 
Golembiewski & Munzenrider, 1973). Other studies have obtained 
the perceptions of subordinates of the trainees (e.g., Burnaska, 
1976; Hand & Slocum, 1970; Snith [study A], 1976). Peers and 
supervisors could also be called upon to rate the trainees. 

One frequently used (but infrequently published) nethod of 
evaluating behaviors is the collection of anecdotes, or stories 
about how the training has been applied. This method has been 
discouraged because of overwhelming biases, but it can be used to 
supplement quantitative ntethcds (e.g., Blakeslee, 1982). 
Individual or group interviews can also be used to gather 
qualitative data, or they can be structured and coded for use as 
quantitative data, as in a study by Byhairi and associates (1976). 

Special Considerations. One major consideration is the 
distinction between actual work behaviors and simulated work 
behaviors. If subjects are evaluated based on their performance 
in contexts which are irregular or with people with whcm the 
subject would not normally ccnciiunicate, the evaluation should be 
regarded as an evaluation of learning, not as an evaluation of 
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behavior on the job* l£ actual work behaviors cannot be dir^fctly 
observed by the re^-earcher, other methods can be used to deuenrdne 
whether work behaviors have been affected by the training. 

Another consideration is the difficulty ii\ measuring 
behavior. Some training professionals lack the background, skill, 
or tiiTte necessary for careful and accurate evaluations on this 
level. An outside consultant may need to be hired for help or 
guidance. 

A third consideration is whether participants are helped to 
transfer their new.ly learned skills to the workplace. No matter 
how actively trainees participate in the training prograiTi and how 
eager they are to try out the new skills, many trainees find it 
difficult to transfer these skills to their jobs (Trost, 1985) • A 
perfectly good training prograiti will have a limited impact on 
behaviors unless participants are helped in applyinj what they 
have learned. 

There are many techniques which could prove useful for 
increasing the transfer of training. In the coHnunication 
training program studied by McNamara and associates (1982) , 
trainers used a review and application log which fostered group 
discussion about organizational applications of what they were 
learning. Another method is to have trainees write down and 
verbally "publish" their goals at the end of the training session. 
Verbally "publishing" a goal is a powerful tool for maintaining 
cGncriitiiient to the goal (Cialdini, 1984). Trainers could also 
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encourage participants to keep journals o£ their efforts to use 
ccnmunication skills at work, in addition, managers and peers can 
be coached in providing positive feedback to trainees. Also, 
follovHup training sessions are popular; sane people consider thent 
to be an essential elenient of a training program design (Trost, 
1985) . 

Another consideration is whether behavior evaluation results 
can be attributed to the training. Before and after measures 
would be needed to distinguish behavior changes made as a result 
of training. A control group would help in determining whether 
these changes might have occurred despite the training. 

Next, we must consider the requirenfients for change. In order 
for behavioral changes to take place, five requirements must be 
met. They are as follows: 1} desire to change, 2) know-how of 
what to do and how-to do it, 3) the right job climate 
(particularly as created by the trainee's manager), 4) help in 
applying the classroom learning, and 5) rewards for changing 
behavior (Kirkpatrick, 1987). 

Finally, it is important to consider the value of 
"statistically significant" behavioral changes. Training programs 
often result in significant behavior changes, but organizational 
behavior management theorists recognize that "scmetimes a 
statistically significant change in behavior may be of little or 
no practical significance to the organization" (Frederiksen, 1982, 
p. 6). If there is no evidence to suggest that trainees' 
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behavioral changes have had a nteasurable impact on organizational 
effectiveness, the results have no practical value to the 
organization. 

Results 

When evaluating at the results level, training professionals 
are looking at the impact of training on organizational 
effectiveness, the "practical significance" of training to an 
organization. Often tiities, the decision of whether to maintain 
training programs depends on their measurable impact on the 
econOTiic bottom line, and necessarily so (Kelley, Orgel, & Baer, 
1984). With the eniphasis organizations place on the bottonn line, 
it seenis sensible to conduct training evaluations at the results 
level, but this is seldom done. 

Variables. A number of variables can be included in an 
evaluation of organizational results. Researchers could watch for 
trends in turnover, absences, sales, custcmer satisfaction, 
quantity or quality of perfontiance, morale, cost reductions, 
grievance reductions, reductions in scrap or errors, accident 
rates, amount of suggestions, new products or new custcmers. 
Other variables could be easily measured, including employee 
satisfaction, working relationships, efficiency in use of time or 
money, or amount of supervision required to perfonri tasks. The 
list is endless. 

Methods. The simplest way to measure results is to monitor 
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theni with methcxis regularly used by the organization. Sane 
comiunication skills training evaluations have looked at such 
indices as sales quotas, performance appraisal ratings, turnover, 
tardiness, and formal grievances (e.g., Ivancevich & McMahon, 
1976; Rosentretter, 1979; Snith [study B] , 1976). Other studies 
have I'sed questionnaires ho measure the improved efficiency of 
time use as a result of intproved communication (Bell & Kerr, 1987} 
or any effects on employee morale (Eltiener & Rye, 1975; ariith 
[study A] , 1976) , on supervisor-employee relationships (Bell & 
Kerr, 1987), on* customer satisfaction (Siiitli [study B] , 1976), and 
on job satisfaction (Ivancevich & McMahon, 1976). 

Special vConsiderations. One major limitation of measurement 
at this level is the difficulty of separating variables. Many 
factors can affect the results of an evaluation, including 
inflation, timing,- location, season, and management 
cooperativeness. If control groups are used and re^i-ults seem to 
be consistent over tirte, however, the evaluations become more 
reliable. 

Another consideration is whether the results to be measured 
are valued by top management. If management is not concerned with 
custcmer satisfaction, for example, an evaluation of this variable 
would be meaningless to the,m, despite the results. Generally, the 
results which are more highly valued are those which are related 
to dollars. 
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Call for Action 

Conniunication skills training )vi.oc">'anis can and V<ave been 
evaluated to detemine thdir effectlveuess. It is important to 
measure the impact j£ each program at various levels. We must 
know how participants feel about the training, what they learned 
from it, how the learning is applied on the job, and how the 
training contributes to overall organizational effectiveness. In 
ooniTiunication skills training, it is especially critical to focus 
on behavior changes and organizational results. 

While billions of dollars are being invested in human 
resource training, it is surprising that training programs are 
rarely evaluated. The lack of training evaluations is a major 
problem, posing a threat to the economic stability of 
organizations and their training departments, and limiting the 
effectiveness of training efforts. To demonstrate accountability, 
a fundamental component of any trai-ying program should be an 
objective assessment of training effects (Burkhart, Hehles & 
Stumphauzer , 1976 ) . 
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REVIEW OF STUDIES 



in this section, a sample of the published coicnunication 
skills training evaluations will be reviev^. An overview o£ the 
levels o£ evaluation used for each study is presented in Table A. 

Bell & K^r (1987) nieasured the effectiveness of a business 
conrcnunication skills training prograiri periodically offered to the 
support and secretarial staff at The University of Texas at 
Austin. The course focused on improving formal contcnunication 
skills (writing, speaking, dictating) and supervisory skills. The 
96 participants in this study met two hours per day, three days a 
week for five weeks, a total of 30 hours. Evaluations occurred at 
all four levels. The participants' reaction to the course was 
evaluated in the typical manner with an end-of -course 
questionnaire, and findings indicated that the participants liked 
the teaching methods, believed the course overall was effective, 
and felt the course was important in perlorming their jobs. At 
the learning level, before-and-after skills tests indicated that 
participants' writing and presentation skills had iiTiproved 
significantly, and a dictation exercise showed that participants 
deitionstrated performance comparable to college senior performance 
in a managerial coninunication course. Behaviors on the job were 
evaluated 3 to 23 months later with a survey questionnaire sent to 
participants and to a matched, randomly selected control group. 
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Table A: Levels o£ Evaluation 





REACTION 


LEARNING 


BEHAVIOR 


RESULTS 


Bell & Kerr, 
1987 


questionnaire 


tests 


questionnaire 


questionnaire 


Bennett, 1981 


X 


simulations & 
observations 


X 


X 


Blakeslee, 1982 


X 


X 


questionnaire 
& anecdotes 


X 


Bowers, et al,, 
1980 


X 


simulations 


X 


X 


Burnaska, 1976 


X 


simulations 


questionnaire 


X 


Byham, et al., 
1976 


X 


X 


interviews 


X 


Clark, et al,, 
1985 


X 


simulations 


X 


X 


Coffinan, 1979 


meetings 


? 


? 


•> 

• 


Elliott, 1978 


written 


tests 


X 


X 


Entener & Rye, 
1975 


questionnaire 


tests 


questionnaire 


questionnaire 


Goleiribiewski & 
Munzenrider, 
1973 


X 


X 


questionnaire 


X 


Hand & Slocum, 
1970 


interv iews 


questionnaire 


questionnaire 


X 


Ivancevich & 
McMahon, 1976 


X 


X 


X 


questionnaire 
& monitoring 


McNaiTiara, et 
al., 1982 


questionnaire 


tests 


(not reported) 


X 


Moses & 

Ritchie, 1976 


X 


siniulations 


X 


X 


Rosentretter , 
1979 


X 


X 


X 


monitor ing 


atiith, 1976 (A) 


X 


X 


questionnaire 


questionnaire 


atiith, 1976 (B) 


^ ■ 


tests 


X 


questionnaire 
& monitoring 


Webb & Howay, 
1987 


questionnaire 


questionnaire 
& tests 


X 


X 



Reaction How did participants feel about the training? 

L earning What did participants learn from the training? 

Behavior How has the learning been applied at work? 

Results How has the training affected the organization? 



21 

25 



Participants reportedly used the skills learned after they returned to 
their jobs, in addition, responses indicated that the training group 
members perfomied more tasks, perfontied a wider variety o£ tasks, and had 
more job responsibilities than manbers of the control group. Finally, the 
evaluation of results, based on the survey questionnaire, indicated that 
participants performed more duties in fewer personnel hours and required 
less of their supervisors* time in routine written communication tasks than 
control. group irtaiibers, and their relationships with their supervisors often 
iniproved. 

Bennett (1981) studied the inipact of human development training on 19 
criminal justice personnel (graduate students) . The course provided 
training in microcounseling, clarification of values, self-knowledge, and 
assertiveness, with an eiriphasis on experiential learning. The impact of 
the course was measured at the learning level. A before-and~a£ter 
microcounseling role-play test indicated that participants had 
significantly improved their ability to give microcounseling responses and 
performed better than a comparison class. Significant improvements in 
microcounseling interview skills were also found in before-and-after 
interviews made with people outside the classroom. The evaluations of 
other aspects of the course are not relevant to this study. 

Blakeslee (1982) examined the effects of communication skills training 
for 164 high- talent professionals in a Fortune 500 manufacturing company. 
The workshop provided training in one-to-one, sitiall group, and group-to- 
group coTrtjunication over 3 & 1/2 days. Evaluation was conducted on the 
behavior level. In a post hoc survey quastionnaire distributed s.x months 
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after completion of the workshop, participants reported statistically 
significant improvement in the quality and quantity of coiifiiunications. 
Anecdotes were also collected to gather infonnation about people's feelings 
of "vAiy" the workshop iiriproved carimunication. 

Bowers, Gilchrist, and Browning (1980) ireasured the effectiveness of a 
two-day course in improving the cawnunication perforrtiance of 70 tax 
enforcenient officers. Evaluation was performed at the learning level. 
Trainees were asked to participate in simulations before, after, and again 
4-6 weeks after the course. Results indicated significant and sustained 
improvement in cawnunication perfontiance. 

Burnaska (1976) evaluated an interpersonal skills modeling program on 
62 middle- level managers at General Electrxc. Evaluations were perfontjed 
at both the learning level and the behavior level. To deterrtiine whether 
learning had occurred and could be sustained, the participants and a 
randomly selected control group were observed in simulated role-plays 
within one month after training and again four to five months later. 
Trainees performed better than the control group at both tinie intervals. 
To deterrtiine effects on the behavior level, onployees of the managers 
completed questionnaires one week prior to training and again four months 
after the training. The eitiployees perceived minimal, if any, changes in 
their managers' behaviors. 

Byharo and associates (1976) studied the influence of modeling training 
on supervisors' ability to handle interactions with their employees. 
Evaluation took place on the behavior level. To determine the effects of 
training, a random sample of trained supervisors' employees participated in 
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highly structured interviews before the training and seven months after the 
training. Subordinates in a roughly matched department were also 
interviewed. Responses were coded and positive differences in employees' 
perceptions of their supervisors after training were found in most 
categories, as well as positive differences in perceptions of trained over 
untrained supervisors. 

Clark and associates (1985) evaluated the effectiveness of an 
interpersonal ski.lls workshop for 27 supervisors in residential programs of 
a hospital for disabled clientele. The workshop was based on a study that 
identified supervisory behaviors which affect employee job satisfaction and 
performance. Evaluation was conducted at the learning level. Trained and 
untrained supervisors were rated on their performance in role-play 
situa'^ions bef-'-.>i' and after training. After training, trained supervisors 
performed significantly better than untrained supervisors and significantly 
better than they had before training. However, the magnitude of change 
frori pre- to post-training, although statistically significant, was not 
grea:. 

Coffinan (1979) presents case studies of evaluations of interaction 
mci:.uC](iR:^jrt lufxieling programs designed to increase the supervisor's skills 
in handling interactions with employees. The procedures for evaluation are 
not airrted at any one level and may or may not be relevant at all levels. 
"Key clients," those people responsible for trainee productivity on the 
job, are asked to gather data on the training impact, then to meet to 
discuss strengths and weaknesses of the training. The training programs ia 
the case study were determined as strong in such areas as "improving 
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working relations and coitiriunications with supervisor and employee" and 
"consistence and uniformity" of training (p. 32) . Vfeaknesses were found in 
such areas as "time in discussion (supervisor and eanployee)" and "too 
arbitrary— does not allow for the human variable." 

Elliott (1978) measured the effects of t-group training on the 
cowTiunication and discrimination skills of 11 counselor trainees (graduate 
students) . The 20-hour training program was evaluated on reaction and 
learning levels. At the reaction level, nine of the eleven participants 
indicated through a subjective, written evaluation that the experience had 
been worthwhile to their personal and professional growth. On the learning 
level, written tests before and after training indicated participants 
significantly improved their canrtunication skills but not their 
discrimination skills. Manbers of a matched control group did not 
significantly improve their communication skills, but did significantly 
improve their discrimination skills (although not significantly more than 
the experimental group). 

Qnener & Rye (1975) investigated the outcome of communications 
awareness training on first-line industrial supervisors. The 40 
participants tiiet 3 hours per day, twice per week for five consecutive 
weeks, a total of 30 hours. Some research was conducted at all four levels 
of evaluation. The reactions to the course, measured by an opinion poll of 
25 of the participants, were generally positive. Overall, participants 
felt the prograni strengthened their supervisory skills and that there was 
an advantage to having an outside instructor. Also, participants indicated 
they did not resent being asl^dd to attend, a continuation of the course 
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would be helpful to all supervisors, and they would not resent being asked 
to attend an extension of the program. On the learning level, written 
tests before and after training indicated significant intprovartent in 
cotinunication skills and discrimination skills. To measure behavioral 
changes, participants were surveyed before and after training. No 
significant changes were four.d in participants' perceptions of their 
conmuni cation styles or supervisory styles. This sane survey also 
indicated, on the results level of evaluation, that there were no 
significant changes in level of morale related to their jobs. 

Golanbiewski & Munzenrider (1973) examined short-term and long-term 
effects of an organizational development program for division managers and 
regional managers of a sales organization. An evaluation was performed on 
the behavior level. Participants were asked to ccRiplete the Likert 
"Profile of Organizational Characteristics" six times during the 245 days 
of the active training program, and again approximately one year after 
termination of the intervention. Results indicated "favorable" lasting 
effects on regional managers in all categories of the Profile, but only 
minimal effects on division managers. 

Hand & Slocun (1970) measured the results of a program attempting to 
improve huirian relations practices of managers in a specialty steel plant. 
The managers attended 1 & 1/2 hour training sessions once a week for 28 
weeks. Evaluations were performed at reaction, learning, and behavior 
levels. Delayed reactions to the training were detenrdned through exit 
interviews of participants which indicated that trainees perceived 
executive staff did not support the program. These perceptions conflicted 
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with perceptions of the executive staff measured by Likert's Profile of 
Organization Characteristics. On the learning level, questionnaires 
cOKipleted by the randomly selected training and control groups before, 
after, and again 90 days after training indicated the following: trainees 
significantly improved in their acceptance of themselves and others upon 
completion of the training, but the change was short-lived. No other 
significant results were found on the learning level. On the behavior 
level, subordinates of the groups completed the Leadership Behavior 
Description Questionnaire before training and again 32 waeks after 
training, indicating significant improvement in trainees' "initiating 
structure" coKiponent, but the program was not considered effective in 
improving trainee behavior overall. 

ivancevich & McMahon (1976) studied effects of a 5-day management and 
organizational behavior training program on 60 first-line supervisors, 46 
sales managers, and 54 engineering supervisors from a large manuf actor ing 
coiipany. Evaluation occurred on the results level. Before, after, and 
again one year after training, participants and a comparison group 
indicated their perceived levels of satisfaction on a questionnaire. Some 
positive differences were indicated in areas of social interaction 
satisfaction, job security and innovativeness satisfaction. Analysis of 
performance ratings suggest some effects on performance, but these effects 
must be accepted with caution. 

McNanara and associates (1982) evaluated a conniiunications training 
program for auditor/evaluator staff at the United States General Accounting 
Office. The four-day course was intended to improve interpersonal problem 
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solving and cottciiunication skills among participants. Evaluations at the 
reaction and learning levels are reported for the 2419 individuals who had 
attended the course so far. Reactions were obtained through questionnaires 
w:nich indicatssd participants held significantly more positive attitudes 
toward the course content, the status of corariunication skills at GAO, and 
the status of personal cowtiunication skills than control group members. 
Also, in an end-of-course questionnaire, participants rated instructor 
conpetence and effort, course materials, and course benefits highly. 
Learning was assessed through a multiple choice achievement test and a not 
too reliable observation exercise, indicating statistically significant 
differen-:es between groups, but these differences were not great. . The 
transfer of training to the job was measured by self-report follow-up 
questionnaires and direct observations of corariunication in work groups, but 
the results were not discussed in this article. 

Moses & Ritchie (1976) examined the impact of supervisory 
relationships training of 90 first-level supervisors in two telephone 
companies. The training followed a behavioral modeling approach and the 
effects were measured on the learning level. Two months after training, 
each trainee and each meniber of a matched, randcxtily assigned control group 
participated in siniulated problon solving discussions, performance ratings 
indicated the trained group's performance was consistently superior to the 
control group's in all three types of probleni solving discussions. 

Rosentretter (1979) exairiined an adult eduction program provided for 68 
department managers of Maryland Cup Corporation. The 15-hour program was 
designed to increase managers' facilitative communication skills for goal 
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setting with subordinates. Evaluation was perfomed at the results level. 
Researchers monitored four econcniic indices (turnover, tardiness, 
performance appraisal ratings, and fonnal grievances) and found significant 
differences only in the turnover index, which was enough to demonstrate a 
total incremental cost savings for the company during the six-month period 
of over $2100. 

SKnith [study A] (1976) studied effects of training designed to improve 
the employee morale at IBM. k group of 18 managers received behavior 
modeling training in two sessions, focusing on "how to coiimunicate 
effectively with individual employees and groups of employees, how to feed 
back opinion survey information, and how to prepare meaningful action plans 
to iniprove morale" (p. 351) . Evaluations were conducted on both the 
behavior level and the results level. To measure effects on behaviors, 
researchers had subordinates of the trainees and of managers in a control 
group complete a itteeting effectiveness questionnaire. The analysis 
indicated that trained managers ran significantly more effective meetings 
with more "open discussion of relevant issues" and "generating good ideas 
for problem solutions" (p. 353). On the results level, opinion surveys 
before and one year after training indicated significant improvements in 
the morale of employees of trained managers and no real change in eniployees 
of control group managers. 

Smith [study B] (1976) studied the effectiveness of different types of 
training which focused on "ways of improving cotimunication with customers, 
customer satisfaction, and sales to customers" (p. 351) . Sixty second- 
level managers participated in this second training program, which was 
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developed as a result of the success of the first training program. 
Effectiveness was evaluated on the learning level and the results level. 
To measure learning, researchers compared managers' written responses to 
custoiier conrtents before and after training, finding significant 
improventents in participants of some training groups (depending on the type 
of training) . On the results level, custcmer satisfaction was measured 
with questionnaires before, four months after, and ten months after 
training, showing a significant, positive relationship between 
conniunication skills of managers after training and later levels of 
customer satisfaction. Also, sales performance was monitored over one year 
and showed an improvement in one training group (modeling plus teant- 
building) . 

Webb & Howay (1987) evaluated the effects of public speaking training 
offered to a siTiall group of secretaries and engineers. The training was 
conducted over four weekly 3-hour sessions and was evaluated on the 
reaction and learning levels. Reactions, measured via end-of-course 
questionnaires, indicated a general positive appraisal of the training, 
particularly regarding the workshop organization and information presented. 
Pre- and post- tests were used to determine what learning had taken place, 
while a pre- and post-survey questionnaire was used to measure levels of 
coTtfTiunication apprehension. No significant differences in rhetorical 
sensitivity or knowledge gain were found, but cowiunication apprehension 
sigificantly decreased. 
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conclusions 

Overall, very few evaluation studies o£ conriiriunication skills training 
have been published. The sairiple reviewed here indicates the following: 

1) Positive effects of communication skills training have been found 
at all four levels of evaluation. 

2) The most popular methods of evaluation have been questionnaires, 
tests, siiTiulations, and monitoring of organizational iiripact. 

3) Evaluation studies typically eniploy more than one method of 
evaluation, thereby strengthening the findings. 

4) Much of the eiiiphasis of evaluations has been on the learning 
level, which is arguably the least relevant or practical to an 
organization. 
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RECOMMENDATIONS FOR RESEARCH 



The task of evaluating the effectiveness of corrinunication 
skills training is critical for deterniining whether the 
organization's needs are being met. The primary intention of this 
study has been to encourage future evaluation efforts by 
demonstrating that evaluations can and have been successfully 
conducted. The results of this study provide a basis for several 
reccnmendations to guide future evaluation efforts. 

First, multi-level evaluations are recomrtended. In order to 
obtain more thorough appraisals of the validity and effectiveness 
of training, evaluations should be conducted at all levels, when 
possible. Two studies which reported evaluations at all levels 
are the Bell & Kerr study (1987) and the Bctiener & Rye study 
(1975) . These studies could be referred to for the guidance of 
future efforts. 

Second, trainers and researchers should continue using a 
variety of methods of evaluation to strengthen the research 
findings. Particularly useful are combinations of quantitative 
and qualitative data collection (i.e., questionnaires and 
interviews) , which result in specific, neasurable responses, plus 
people's feelings about those responses. 

Third, techniques for accurately rtieasuring behavior changes 
need to be developed. Seme researchers have siirtulated 
environments to deterrtiine whether behaviors have been learned, but 
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these methods do not denionstrate whether behaviors are applied 
outside the simulated enviroments. Practical methods for 
directly observing coirniunication behaviors at work need to be 
developed. 

Fourth, more studies to determine the training's ultimate 
inipact on the organization ought to be developed. To support the 
assertion that improved communication can result in improved 
organizational efficiency and effectiveness, a variety of 
organizational factors should be carefully examined. 

Fifth, the long-term effects of training must be examined. 
Seme studies have found training effects to be short-lived (e.g.. 
Hand & Slocum, 1970) . As new techniques and follow-up programs 
are developed to prolong the effects of training, long-term 
evaluations can be simultaneously conducted. 

Finally, evaluations should be published. Current evaluation 
efforts can serve as models for future efforts, but they need to 
be readily accessible to the public. In addition, as more 
contmunication training evaluation studies are published, the 
credibility of coitiriunication skills training in organizations will 
be enhanced.. 
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